controlled experiment
Looking Beyond Single Images for Contrastive Semantic Segmentation Learning - Supplementary Material - 1 Additional results 1.1 Controlled experiment on auxiliary label generation
Table 1 reports the results of a controlled experiment evaluating different components in our framework for auxiliary label generation. Positive correspondences are generated by matching pixels across different augmentations of the same image. With respect to the clustering algorithm, K-means performs better than DBSCAN (#4 vs. #5), which is We show qualitative results, comparing different feature extractors in Figure 1. DBSCAN is limited by the memory and computational complexity. Corresponding qualitative results are shown in Figure 3. Tables 3-5 show We observe the best performance when 5% outliers are removed.
- Research Report > Strength High (1.00)
- Research Report > Experimental Study (1.00)
Inappropriate Benefits and Identification of ChatGPT Misuse in Programming Tests: A Controlled Experiment
Toba, Hapnes, Karnalim, Oscar, Johan, Meliana Christianti, Tada, Terutoshi, Djajalaksana, Yenni Merlin, Vivaldy, Tristan
While ChatGPT may help students to learn to program, it can be misused to do plagiarism, a breach of academic integrity. Students can ask ChatGPT to complete a programming task, generating a solution from other people's work without proper acknowledgment of the source(s). To help address this new kind of plagiarism, we performed a controlled experiment measuring the inappropriate benefits of using ChatGPT in terms of completion time and programming performance. We also reported how to manually identify programs aided with ChatGPT (via student behavior while using ChatGPT) and student perspective of ChatGPT (via a survey). Seventeen students participated in the experiment. They were asked to complete two programming tests. They were divided into two groups per the test: one group should complete the test without help while the other group should complete it with ChatGPT. Our study shows that students with ChatGPT complete programming tests two times faster than those without ChatGPT, though their programming performance is comparable. The generated code is highly efficient and uses complex data structures like lists and dictionaries. Based on the survey results, ChatGPT is recommended to be used as an assistant to complete programming tasks and other general assignments. ChatGPT will be beneficial as a reference as other search engines do. Logical and critical thinking are needed to validate the result presented by ChatGPT.
- Research Report > Strength High (1.00)
- Research Report > Experimental Study (1.00)
Injecting Domain Knowledge in Neural Networks: a Controlled Experiment on a Constrained Problem
Silvestri, Mattia, Lombardi, Michele, Milano, Michela
Given enough data, Deep Neural Networks (DNNs) are capable of learning complex input-output relations with high accuracy. In several domains, however, data is scarce or expensive to retrieve, while a substantial amount of expert knowledge is available. It seems reasonable that if we can inject this additional information in the DNN, we could ease the learning process. One such case is that of Constraint Problems, for which declarative approaches exists and pure ML solutions have obtained mixed success. Using a classical constrained problem as a case study, we perform controlled experiments to probe the impact of progressively adding domain and empirical knowledge in the DNN. Our results are very encouraging, showing that (at least in our setup) embedding domain knowledge at training time can have a considerable effect and that a small amount of empirical knowledge is sufficient to obtain practically useful results.